Mixtures of Gaussians

This tutorial treats mixtures of Gaussian probability distribution functions. Gaussian mixtures are combinations of a finite number of Gaussian distributions. They are used to model complex multidimensional distributions. When there is a need to learn the parameters of the Gaussian mixture, the EM algorithm is used. In the second part of this tutorial mixtures of Gaussian are used to model the emission probability distribution function in Hidden Markov Models. Usage To make full use of this tutorial you should 1. Download the file MixtGaussian.zip which contains this tutorial and the accompanying Matlab programs. 2. Unzip MixtGaussian.zip which will generate a subdirectory named MixtGaussian/matlab where you can find all the Matlab programs. 3. Add the folder MixtGaussian/matlab and the subfolders to the Matlab search path with a command like addpath(’C:\Work\MixtGaussian\matlab’) if you are using a Windows machine or addpath(’/home/jack/MixtGaussian/matlab’) if you are using a Unix/Linux machine. 1 Mixtures of Gaussians 1.1 Formulas and Definitions Gaussian Mixtures are combinations of Gaussian, or ‘normal’, distributions. A mixture of Gaussians can be written as a weighted sum of Gaussian densities. Recall the d-dimensional Gaussian probability density function (pdf): g(μ,Σ)(x) = 1 √ 2π d√ det (Σ) e− 1 2 (x−μ) TΣ−1(x−μ), (1) with mean vector μ and covariance matrix Σ. A weighted mixture of K Gaussians can be written as